Traversal times of Markov processes

Traversal times of Markov Processes by R. v. Chacon and Benton Jamison

There are two main results we wish to state which are proved in a series of papers starting with [1]. The first is that whenever a particle which is undergoing motion governed by a strong Markov process with stationary transition probabilities and without holding points traces out a fixed trajectory segment, it must trace it out in a fixed length of time depending only on the transition probabi...

Stochastic transcription and traversal times of Markov chains

Markov Decision Processes with Constrained Stopping Times

The optimization problem for a stopped Markov decision process is considered to be taken over stopping times constrained so that E 5 for some xed > 0. We introduce the concept of a randomized stationary stopping time which is a mixed extension of the entry time of a stopping region and prove the existence of an optimal constrained pair of stationary policy and stopping time by utilizing a Lagra...

Extinction times in Multitype Markov Branching Processes

In this paper, a distributional approximation to the time to extinction in a subcritical continuous-time Markov branching process is derived. A limit theorem for this distribution is established and the error in the approximation is quantified. The accuracy of the approximation is illustrated in an epidemiological example. Since Markov branching processes serve as approximations to nonlinear ep...

Relative entropy and waiting times for continuous-time Markov processes

For discrete-time stochastic processes, there is a close connection between return/waiting times and entropy. Such a connection cannot be straightforwardly extended to the continuous-time setting. Contrarily to the discrete-time case one does need a reference measure and so the natural object is relative entropy rather than entropy. In this paper we elaborate on this in the case of continuous-t...